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1 .  Introduction 


In  a  recent  paper  [  1 1  ]  published  by  the  first  author,  it  was  suggested 
that  many  problems  of  image  restoration  are  probably  geometric  in  charac¬ 
ter  and  admit  the  following  initial  linear  formulation:  The  original  f  is  a 
vector  known  a  prior  to  belong  to  a  linear  subspace  9^  of  a  parent  Hilbert 

space  h,  but  all  that  is  available  to  the  observer  is  its  image  P  f,  the 

a. 

projection  of  f  onto  a  known  linear  subspace  9  (also  in  W). 

a 

1)  Find  necessary  and  sufficient  conditions  under  which  f  is  uniquely 

determined  by  P  f  and  2)  find  necessary  and  sufficient  conditions  for  the 
a 

stable  linear  reconstruction  of  f  from  P  f  in  the  face  of  noise.  (In  the  later 

a 

case,  the  reconstruction  problem  is  said  to  be  completely  (or  well)  posed.) 
The  answers  turn  out  to  be  remarkably  simple. 

a)  f  is  uniquely  determined  by  Paf  iff  9^  and  the  orthogonal  comple¬ 
ment  of  9  have  only  the  zero  vector  in  common. 

a 

b)  The  reconstruction  problem  is  completely  posed  iff  the  angle 

between  9  and  the  orthogonal  complement  of  9  is  greater  than  zero. 
d  a 

(All  angles  lie  in  the  first  quadrant.) 

c)  In  the  absence  of  noise,  there  exists  in  both  cases  a)  and  b)  an 
effective  recursive  algorithm  for  the  recovery  of  f  employing  only  the 
operations  of  projection  onto  9^  and  projection  onto  the  orthogonal  comple¬ 
ment  of  9^.  These  operations  define  the  necessary  instrumentation. 

This  point  of  view  has  proved  to  be  very  fruitful  and  can  be  made  to  en¬ 
compass  many  important  applications  [12], 

However,  it  appears  that  a  linear  formulation  is  only  achievable  by 
discarding  information  concerning  the  original  f,  and  as  a  result  the 
associated  restoration  problem  is  often  ill-posed  til.  Theorem  2].  The 
entire  process  of  smoothing  can  be  conceptualized  as  a  technique  for  re¬ 
introducing  the  missing  information  and  therefore  plays  an  essential  role 
in  combatting  the  effects  of  noise. 

As  is  shown  in  detail  in  Part  II,  a  linear  image  restoration  problem 
transforms,  under  smoothing,  into  a  nonlinear  one  of  the  following  kind: 

The  original  f  is  known  a  priori  to  belong  to  the  intersection  CQ  of  m 


1 


(1.1) 


well-defined  closed  convex  sets  C^,  I*.,,  C^;  i.e.. 


m 


m 

fceft*  n  c 

0  i=i  1 


Given  only  the  projection  operators  P.  onto  the  individual  C/s,  i  =  Hm, 
restore  f,  preferably  by  an  iterative  scheme.  Thus,  the  P/s  define  the 
necessary  instrumentation  in  an  arbitrary  Hilbert  space  setting.  We  are 
now  in  a  position  to  summarize  the  contents  of  Part  I. 

Section  2  provides  some  basic  tutorial  material  on  convex  sets  and 
projection  operators  onto  closed  convex  sets.  Section  3  is  devoted  to  an 
in-depth  leisurely  study  of  nonexpansive  maps  and  their  fixed  points,  with 
special  emphasis  on  iterative  methods.  Because  of  the  importance  of 
these  methods,  we  have  supplied  full  proofs  in  order  to  make  the  report 
self-contained.  Section  4  applies  the  theorems  in  Section  3  to  the  problem 
of  finding  points  belonging  to  the  intersection  of  a  finite  number  of  closed 
convex  sets.  It  is  shown  that  the  latter  is  a  particular  case  of  the  problem 
of  finding  a  fixed  point  of  an  asymptotically  regular  nonexpansive  operator 
by  iteration,  hi  general,  the  convergence  of  this  iteration  is  only  weak  and 
the  question  of  strong  convergence  is  examined  in  some  detail.  In  this  re¬ 
gard,  Theorem  4.2  is  a  major  result  which  serves  as  the  starting  point  for 
Part  II. 


Specifically.  Section  5  in  Part  II  pursues  the  consequences  of  identify¬ 
ing  the  process  of  smoothing  with  that  of  "opening"  up  closed  convex  sets  by 
enlarging  them  to  possess  interiors .  Section  6  develops  algorithms  for  the 
realization  of  several  important  projection  operators  onto  closed  convex 
sets,  and  Section  7  describes  some  numerical  results.  Lastly,  Section  8 
concludes  with  some  observations  and  suggestions  for  future  research. 


I 


2.  Some  Properties  of  Convex  Sets  in  Hilbert  Space 


Definition  2.  1:  A  subset  C  of  M  is  said  to  be  convex  if  together  with 
any  Xj  and  x 2  it  also  contains  px,  +  (l-p)x2  for  all  p,  0<  p<  1. 

Theorem  2.  1 :  Let  C  denote  any  closed  convex  subset  of  M  and  let  f 
be  any  element  of  h.  Then,  there  exists  a  unique  x^eC  such  that 


inf  f-x  =  f-x. 


(Clearly,  x^  is  the  element  in  C  closest  to  f  in  norm.) 

Proof.  Let  us  first  demonstrate  uniqueness.  Suppose  that 

inf  ||  f-x  |  =  6  =  ||f-x0|j  =  ||f-y0|| 
xeC 

where  x^eC  and  y^eC.  Then,  by  replacing  x  and  y  in  the  identity 

x  2  2  II  x  II  2  +  ||  y  H  2 

11^#  ♦«2?ll  = 

by  f  -Xq  and  f-yQ,  respectively,  we  obtain 

ll<-^l|2  =  62-||!°Zo»V  . 


(2.1) 


(2.2) 


(2.3) 


(2.4) 


But,  because  C  is  convex. 


Vy0 


eC  so  that 


xo+y0  .*2 


(2.5) 


whence,  ||x0-y0||  =  0  and  xQ=  yQ. 


From  the  definition  of  infimum,  there  exists  a  sequence  {x^}  of 


elements  contained  in  C  such  that 

i|f-*nll  =  6  * 

By  replacing  x  and  y  in  (2.3)  by  f-x^  and  f-x^  respectively,  we  get 


(2.6) 


■>  x  +x  2 

l“_  ullf  —  ii  2  .  it  ,  ..  it  c.  .  I)  .  n  m  ir 


flxn'XnJ  =2<llf-xnH  +  llf"xnJ  )_4lif 


•  (2.7) 


I 


f 


X  +x 

Thus,  since  — "g-  —  eC.  for  n,m=  l-»»t  (2.5)  and  (2.7)  yield 

0<  lim  ||x  -x ||2  <462^  462=  0  (2.8) 

n,  m  -*  oo  n  111  “ 

which  implies  that 

Urn  |x  -x  ||=0  .  (2.9) 

n,m-*oo 

The  sequence  (xn)  is  therefore  Cauchy  and  converges  to  a  limit  x^cC  be¬ 
cause  C  is  closed.  From  (2.6), 

||f-x0H  =  6  ,  (2.10) 

Q.E.D. 

Theorem  2.  1  is  fundamental  and  leads  immediately  to  the  notion  of 
a  projection  operator:  For  any  x£h  the  projection  P^x  of  x  onto  C  is  the 
element  in  C  closest  to  x.  As  we  have  seen,  if  C  is  closed  and  convex, 
PpX  exists  and  is  uniquely  determined  by  x  and  C  from  the  minimality 
criterion 

|Jx-P-x||  =  min||x-y||  .  (2.11) 

C  yeC 

This  rule,  which  assigns  to  every  xeli  its  nearest  neighbor  in  C,  defines 
the  (in  general)  nonlinear  projection  operator  P^rM-^C  without  ambiguity. 
Later  we  shall  develop  several  alternative  important  characterizations  of 


Of  course  any  CLM  is  automatically  convex  and  closed  and  the 
corollary  that  follows  is  the  classical  orthogonal  projection  theorem. 

Corollary  1:  Let  G  denote  any  CLM  in  W.  Then,  every  x€M  possesses 
a  unique  decomposition  of  the  form  x  =  x^+x^  where  Xj  =  PqXj€G  and 
x^exG,  the  orthogonal  complement*  of  G  . 

Proof.  Suppose  that  x=  x^+ x^  =  x^  +  x^  where  x^  and  x^  belong  to  G 
and  x^  and  x^  belong  to  xG.  Then,  Xj  -x^  =  x^-x^  *8  a  member  of  both  G 

* For  any  linear  manifold  G,  yexG  iff  (x,y)  =  0  for  all  xeG.  It  is  easily 
shown  that  xG  is  always  a  CLM  and  that  if  G  is  a  CLM,  then  x(xG)=  G. 
Evidently,  GH(xG)  =  ($ 3  and  Gcx( xG). 
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and  aG.  Thus,  from  the  previous  footnote,  x^x^,  x^  =  x^  and  the  decom¬ 
position,  if  it  exists,  must  be  unique. 

Choose  Xj  =  PqX  and  set  x^x-P^x.  Then,  x  =  Xj+x^  and  it  only  re¬ 
mains  to  show  that  x^  is  orthogonal  to  G.  That  is,  (x^,y)  =  0  for  all  yeG. 
From  the  definition  of  P^.  and  the  convexity  of  G,  for  any  (J,  0<  |i  <  1, 
py+(l-U)PGxeG  for  every  yeG  so  that 

||x-PQx||Z<  |Jx-Hy-(l-H)PgX || 2  (2.12) 

Or,  upon  expansion  of  the  right-hand  side  of  (2.  12), 

J|x-PGx]|2<  ||x-Pgx||2+ ^2|Jy-PQx|j2  -  2MRe(x-PGx,y-PQx)  .(2.13) 

Hence, 

2Re(x-PGx,y-PGx)  <  |i  ||y-PGx||2  (2.  14) 

and  by  letting  |i-»0  we  find  that 

Re(x-PGx,  y-PGx)  <  0  (2.15) 

for  all  yeG.  Now,  since  G  is  also  a  linear  manifold  it  contains  \y  for  every 
real  X  and  (2.  14)  yields,  for  X  real, 

XRe(x-PGx,  y)  <  (x-PqX,  PGx)  .  (2.16) 

This  inequality  can  only  be  true  if 

Re(x-PGx,  y)  =  0  (2.17) 


for  all  yeG.  If  now  y  in  (2.  17)  is  replaced  by  iy,  we  get 

Im(x-PGx,  y)  =  0 

so  that  finally 

(x-PQx,  y)  =  0  ,  y€G  , 


Q,  E.  D, 


(2.18) 

(2.19) 
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Corollary  2:  Let  C  be  a  closed  convex  set.  Then,  for  any  xeW, 


Re(x-Pcx,  y-P^x)  <  0  {2. 20) 

for  every  yeC.  Conversely,  if  some  zeC  has  the  property 

Re(x-z, y-z)  <  0  (2.21) 


for  all  y  eC,  then  z  =  P^x. 

Proof.  A  review  of  the  proof  of  corollary  1  reveals  that  (2.  15)  is 
valid  for  any  closed  convex  set  C.  Conversely,  (2.21)  implies  that 

||x-y  1  2=  ||x-z+z-y  ||  2=  || x-z  Jj  2-  2Re(x-z,  y-z)+ 1|  z-y  ||  2>|| x-z  || 2  (2.  22) 

for  all  yeC.  Thus,  by  Theorem  1,  z  =  P^x,  Q.E.D. 

Corollary  3:  Any  closed  bounded  convex  set  C  is  weakly  compact. 

Proof.  What  we  have  to  show  is  that  from  every  sequence  {xn)cC 
it  is  possible  to  extract  a  subsequence  {x^/}  that  converges  weakly^  to 
some  limit  f,  and  that  all  such  weak  limits  belong  to  C.  By  hypothesis,  C 
is  bounded  as  is  therefore  every  sequence  {x^}  contained  in  C.  Hence, 
since  Hilbert  space  M  is  weakly  compact,  there  exists  a  subsequence 
(xn/)  of  {x^}  such  that  x^/  -»  f ,  fe  M.  Now  according  to  (2.  20), 

Re(f-Pcf ,  xn/  -  Pcf)  <0  (2. 23) 

for  all  n*.  Thus,  x^/  -»  f  implies  that 

0>Re(f-Pcf,  f-Pcf)  =  ||f-Pcf  1  2>  0  (2.  24) 

or,  )|f-Pgf|j=0.  Consequently,  f=P^feC,  Q.E.D. 

The  proof  of  corollary  3  reveals  that  a  closed  convex  set  C,  bounded 
or  unbounded,  contains  all  its  weak  limits  and  is  therefore  weakly  closed. 
Now,  since  a  strong  limit  is  automatically  weak,  it  is  clear  that  weak 
closure  of  any  set  implies  its  strong  closure  but  the  converse  is  in  general 
false.  From  this  point  of  view,  convex  sets  appear  to  be  quite  exceptional. 

- 2 - 

This  means  that  lim(g,xn/)  =  (g,f)  for  every  g€)t. 
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In  a  real  Hilbert  space,  the  inequality  (2.  20)  for  closed  convex  sets 
C  reads 

(x-PgX,  y-P^x)  <  0  ,  allyeC  .  (2.25) 


In  this  guise  it  can  be  interpreted  to  mean  that  the  vector  x-P^x  is  support¬ 
ing  to  C  at  the  point  P^xeC.  As  Fig.  2.  1  suggests,  x-P^x  is  "normal"  to 
the  "tangent  plane"  to  C  erected  at  the  point  P^x.  This  plane  has  C  and  x 
on  opposite  sides  and  therefore  separates  one  from  the  other.  Note  also 
that  the  angle  vp  between  the  vectors  x-P^x  and  y-P^x  is  never  less  than  90°. 


Fig.  2.  1.  C  is  a  closed  convex  set. 

Theorem  2,  2 :  Let  C  be  any  closed  convex  set.  Then,  for  every 
^air  of  elements  x  and  y  in 


||  Pcx-Pcy  ||  2  <  Re(x-y,  PgX-P^) 


(2.26) 


that 


Proof. 


Since  P^x  and  P^,y  both  belong  to  C  it  follows  from  (2.20) 


Re(x-Pcx,  P^y-P qX)  <  0 


(2.27) 


and 


Re(y-Pcy,  Pcx-Pcy)  <  0 


(2.28) 


Thus,  (2.26)  is  obtained  immediately  by  addition,  Q.E.  D. 
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Corollary  1:  Projection  operators  onto  closed  convex  sets  C  are 
nonexpansive  and  therefore  continuous. 

Proof.  Schwartz's  inequality  applied  to  (2.26)  yields,  for  every  x 
and  y  in  M, 

||Pcx-Pcy|l  <  || x-y ||  .  (2.29) 

Therefore,  under  the  operator  P^,  the  distance  between  two  images  never 
exceeds  the  distance  between  the  two  originals  which  is  precisely  the  defi" 
niti on  of  nonexpansivity.  A  fortiori,  ||x-y||  "small"  implies  ||p^,x-P^y|| 
"small"  so  that  P^  is  continuous,  Q.E.D. 

It  is  possible  to  strengthen  (2.29)  in  various  directions  by  imposing 
one  or  more  constraints  on  the  "curvature"  of  C.  Admittedly,  this  strengthen¬ 
ing  appears  more  natural  in  a  finite -dimensional  setting  and  leads  to  several 
technical  refinements  not  fully  exploited  in  this  report.  Nevertheless,  we 
present  the  results  for  possible  future  applications  and  because  their  proofs 
depend  on  Theorems  2.  1  and  2.  2  in  an  essential  way. 

Definition  2.2:  A  convex  set  C  is  said  to  be  strictly  convex  if  x  EC, 
yeC  and  x^y  imply  that  (xry)/2,  the  midpoint  of  the  "chord"  connecting  x 
and  y,  is  an  interior  point  of  C. 

Definition  2.3:  A  convex  set  C  is  said  to  be  uniformly  convex  if  there 
exists  a  function  6(t)  positive  for  T>0,  and  zero  only  for  T  =  0,  such  that 
x, y eC  and 

||z  .  ^+X||  <6(||x-y|)  (2.30) 


imply  zeC. 

Definition  2.4:  A  uniformly  convex  set  C  for  which  it  is  possible  to 
choose 

6(t)=mtZ  ,  (2.31) 

u  a  positive  constant,  is  said  to  be  strongly  convex.  (Evidently,  strong 
convexity  implies  uniform  convexity  which  implies  strict  convexity.) 

Theorem  2.3  [  1  ]:  If  C  is  strictly  convex,  y€C,  x<C,  y^P  £,x,  we  have 

Re(x-P^,x,  y-P^x)  <  0  .  (2.32) 
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Proof.  By  definition,  if  yeC  and  P^x^y,  there  exists  an  e>0  (which 
depends  in  general  on  the  choice  of  x  and  y),  such  that 

Pcx+y 

w  =  *  + 2 — eC  (2.33) 


for  all  z  that  satisfy  ||z||  <  e.  Hence,  if  x^C  we  can  choose 

e(x-Pcx) 

llx-pcxll 

and  then  replace  y  in  (2.20)  by  w  to  obtain 

Re(x-Pcx,y-Pcx)  <  -2e||x-Pcx||  <  0  , 


(2.34) 


(2.35) 


Q.E.D. 

Corollary  3:  If  C  is  uniformly  convex,  yeC  and  x^C,  then 

Re(x-Pcx,y-Pcx)<  -26(||y-Pcx||)  .  ||x-Pcx|!  .  (2.36) 

If  C  is  strongly  convex, 

llpcx-yi!  pHx-yll  (2.37) 

where 

p  =  (l+2p||x-P(,x||)"1  .  (2.38) 

Proof.  Observe  first  that  if  y=Pcx,  (2.35)  is  correct  with  e=0.  Con¬ 
sequently,  in  view  of  the  properties  of  6(t),  (2.36)  follows  immediately  by 
substituting  6(|iy-Pcx||)  for  C  in  (2.35).  To  derive  (2.37)  we  rewrite  (2.35) 
in  the  form 

2 

llPCX"yll  £Re(x-y,pcx-y)  -2e||x-pcx||  ,  (2.39) 

and  then  use  Schwartz's  inequality  to  obtain 

llPCx-y||2-|lPcx'yll  ‘  Ux“yl +2elix-pCxll  <  0  •  (2.40) 

Or,  with  e  replaced  by  |i||p^x-y||  , 


9 


!!pc*-y||(||Pcx-y||  +2u|Pcx-y||  .  ||x-Pcx||  -  ||x-y  ||)  <  0 


(2.41) 


so  that  always  , 


llx~y|l 

1  +  2u||x-Pcx|| 


(2.42) 


Q.E.D. 

The  closed  half-circle  C  shown  in  Fig.  2.2a  is  convex  but  not  strictly 
convex  because  the  midpoint  z  is  not  an  interior  point.  On  the  other  hand, 
the  full  closed  circle  in  Fig.  2.  2b  is  the  prototype  of  a  strongly  convex  set. 


(a)  Closed  half-circle.  (b)  Closed  full-circle. 


Figs.  2.2a  and  2.2b. 
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3.  Nonexpansive  Maps  and  Their  Fixed  Points -Basic  Theorems 


Definition  3.1:  A  mapping  T:  C-*  K  is  said  to  be  a  contraction  if  there 
exists  a  positive  constant  8,  0<  0  <  1,  such  that 

Jj Tx-Ty  1|  <  8 i| x-y H  (3.1) 

for  all  x,  y  eC. 

Any  contraction  has  at  most  one  fixed  point.  For  if  Tx^  =  x^  and 
Tx^  =  x^,  then 

IJxj-x^H  =  HTXj-Tx^H  <  0||xj-x^||  (3*2) 

which  implies  Ijx^-x^H  =  0  and  hence  x^=x^. 

Theorem  3.1:  (The  contraction  principle).  If  C  is  a  nonempty  closed 
subset  of  M,  any  contraction  mapping  T  of  C  into  itself  possesses  a  unique 
fixed  point  x^.  Moreover,  starting  from  any  element  x^  of  C,  Tnx^ "^x^ 
as  n  -*<». 

Proof.  Since  we  already  know  that  there  is  at  most  one  fixed  point, 
we  need  only  show  that  a  fixed  point  exists  and  is  obtainable  by  iteration. 
Let  Xq  be  any  member  of  C  and  set 

x  =  Tx  .  ,  n=l-»oo  .  (3.3) 

n  n-i  ' 

Then, 

I! xn+ 1 "xn II  —  0ilxn"Xn-ltt- *  *  <3-4) 

so  that  for  any  m  >n 

II X  -X  H  <  ||  X  -X  ,||  +  ||x  .  -X  ,||  +  .  .  .  +||x  .  .  -X  I 

11  m  nn  — 11  m  m-1"  "  m-1  m-Z"  "  n+1  n" 

<  ePo+e*  • .  •  +em'n’1)||x1-x0|| 

<  ©"(  1  -  6) “ 1  Uxj -xQ  ||  .  (3.5) 

Since  0<  8<  1,  it  follows  that  [x^}  is  a  Cauchy  sequence.  Therefore, 
as  C  is  closed,  it  converges  to  an  element  x^eC.  But  clearly, 
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(3.6) 


II  Tx«r*oo  II  <  H  Txoo-xn+ 1 1  +  H  Xn+ 1  ’Xpo  I 

<  ellxoo-x„ll+l!xoo-Xn+lH 

and  the  right  side  tends  to  zero  as  n  -»°o.  Thus,  IjTx^-x^JI  =  0,  i.e. , 

Tx  =x  .  It  should  also  be  noted  that  (3.  5)  yields  the  estimate 
oo  oo 

llxoo-xnU-  0,1(1  ■0)'1HX1"XO  II  *  (3,7) 

Q.E.D. 

Corollary:  Let  T£  ,  i  an  integer  >1,  be  a  contraction  of  a  nonempty 
closed  set  C  into  itself.  Then,  T  possesses  a  unique  fixed  point  xm  obtain¬ 
able  as  the  limit  of  any  sequence  {TnXpV,  XqCC  . 

Proof.  According  to  Theorem  3.1,  there  exists  a  unique  x^eC  such 
that  T£xoo  =  xoQ.  Hence, 

Txoo  =  T(Tixoo)=Ti(Txoo)  (3.8) 


and  Tx  eC  is  also  a  fixed  point  of  T£.  By  uniqueness,  Tx^x^  and  is 
therefore  a  fixed  point  of  T.  Consider  the  sequence  {T  xq}  as  n-*<»  and 
write  n=qi+r  where  q  is  an  integer  and  0<  r  <1  .  Then,  as  n-*°of  q-*oo  and 


-x 


00 


11  =  «T 


rTq£x  -Trx 
1  x0  oo 


li 


-X 


00 


II 


■*  o  . 


Consequently,  T^^x^  as  n-»o°,  Q.E.D. 

Contraction  is  a  very  strong  requirement  to  impose  on  a  mapping. 

A  weaker  notion  is  that  of  "shrinking". 

Definition  3.2:  A  mapping  T:  C-*  H  is  said  to  be  shrinking  if 

flTx-Tyfl  <  ||x-y||  ,  x^y  ,  (3.9) 

for  all  x, y eC. 

3 

Theorem  3.  2:  A  shrinking  map  T  of  a  compact  set  C  into  itself  always 
possesses  a  fixed  point. 

3  A  subset  C  of  H  is  said  to  be  compact  if  every  sequence  of  elements  in  C 
has  a  subsequence  which  converges  to  a  limit  element  in  C. 
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Proof.  The  numerical  function  ||Tx-x||  is  obviously  continuous  on 
C.  Hence,  since  C  is  compact  there  exists  an  element  x^eC  such  that 

infllTx-xIMlTx^-xJ  .  (3.10) 

xeC 

But  we  must  have  Tx  =  x„  because  otherwise 

oo  oo 

|T(TxJ-TkJ  <  llTx^-x^U  ,  (3.11) 

a  contradiction,  Q.E.  D. 

In  many  applications  even  shrinking  is  difficult  to  achieve  and  our 
final  weakening  of  the  contraction  idea  is  embodied  in  the  concept  of  a  non- 
expansive  operator  (mapping). 

Definition  3.3:  A  mapping  T:  C-*)i  is  said  to  be  nonexpansive  if 

|| Tx-Ty ||  <  || x-y ||  (3.12) 

for  all  x,  y  cC. 

Theorem  3.3:  Let  T:  C~»C  be  a  nonexpansive  map  whose  domain  C  is 
a  nonempty  closed  bounded  convex  set.  Then,  T  has  at  least  one  fixed  point. 

Proof,  Let  y^  be  any  preselected  member  of  C  and  let  the  set 
Cq={x:  x=y-yg,yeC  }.  This  translate  Cq  is  evidently  also  a  closed  bounded 
convex  set  which  moreover  contains  the  zero  vector  <j>.  Clearly,  every 
x€Cq  possesses  a  unique  decomposition  x=y-yp,  yeC.  Let  F:  Cq-*Cq  be  de¬ 
fined  by 

Fx  =  Ty  -yQ  .  (3.  13) 

This  map  F  is  nonexpansive  because  x^  =  Yj"yQ  an^  *mP*y  that 

Ufxj-fx^I  =  U Tyi-Ty2||  <  ||yi-y2||  =  ll(y1-y0)-(y2-y0) II  =  ll’v^ll  . 

(3.  14) 

For  any  fixed  k,  0<  k<  1,  the  map  G=  kF  is  a  contraction  of  Cq  into 
itself.  Indeed,  for  any  x€Cq,  kFx=  k(Fx)  +  (1 -k)<|>eCQ  and  for  all  x^.x^cCq, 


13 


(3.15) 


llGXj-Gx^l  =k||Fx1-Fx2U  <  kllx1-x2ll 

Hence,  invoking  Theorem  3.  1,  there  exists  for  every  constant  k,  0<k<l, 
a  unique  x^Cq  such  that 

Xk=kFxk  *  (3.16) 

If  it  can  now  be  shown  that  xk*»g  as  k -*  1  from  below,  then  by  the  continuity 
of  F  and  gCCg,  it  is  seen  from  (3.16)  thatg  =  Fg.  Or,  since  g  s  f-yQ,  feC, 
we  obtain  f  =  Tf  so  that  f  is  a  fixed  point  of  T.  We  shall  actually  prove  that 

K®  xk=g  (3.17) 

k-*  1  ;0<  k<  1  K 

where  g  is  the  (unique)  fixed  point  ot  F  in  CQ  of  minimum  norm. 

Assume  that  0  <  k  <f  <  1 ,  xk=kFxk>  x^  =  iFx^  and  let  h -  x^  -x^.  Then, 
since  || Fx^ -Fx^||  <  ||x^  -x^|| ,  we  obtain 

(*  _1(xk+h)  -k"1^  ,  i_1(xk+h)  -k_1xk)  <  || h  ||  2  (3.18) 

or, 

(^"1"k"1)2||xk||2+(i"2-l)  ||  h  || 2  <  Zl  (k"1-*  _l)Re(xk.  h)  .  (3.  19) 

Thus, 

Re(xk>  h)  >  0 

which,  together  with  the  identity 

11^  II  2  =  llxk+hIl2  =  llxk|2+  ||h||2+2Re(xk,h)  , 
leads  to  the  inequality 

llxiliZ>IIxkl|2+  llx|-xkU2  •  (3.2  2) 

To  sum  up,  for  any  choice  of  sequence  0<k1<  k^  <  ...  such  that  k.-*l,  the 
sequence  1  |j xk  ||  }  is  monotone  nondecreasing  and  bounded  (because  Cq  is 
bounded).  It  therefore  converges  and,  in  particular,  from  (3.22), 


(3.20) 


(3.21) 
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Nxi”xiJ2-  Hxi!l2"  Hxkll2  ** 0 


(3.23) 


as  i,  k-»°o.  By  the  completeness  of  W,  Xj^-»geCQ  because  Cq  is  closed. 

(Of  course,  the  limit  g  is  independent  of  the  particular  selection  of  sequence 


Lastly,  let  e  be  any  fixed  point  of  F  in  Cq. 


lhen,  e=l*Fe  and  we 


can  apply  (3.22)  with  x^=e ,  i=l,  x^=x^  and  k=k^  for  any  i  =  1  -» oo.  As  i-»oof 


x^  **g  so  that 
l 


(3.24) 


Therefore,  ||g  ||  =  inf  ||e  j|,  as  e  ranges  over  the  fixed  points  of  F  in  CQ,  Q.E.  D. 

Theorem  3. 3  is  due  to  Browder  [23  but  the  proof  we  have  presented  is, 
in  all  its  essentials,  that  given  by  B.  Halpern  [3  ].  Unlike  Theorem  3.2 
(which  it  does  not  subsume),  the  compactness  assumption  on  C  is  dispensed 
with  completely  and  replaced  instead  by  the  much  weaker  constraints  of 
convexity  and  boundedness.  Nevertheless,  in  many  signal-processing  ap¬ 
plications  even  this  assumption  of  boundedness  may  be  questionable  because 
accurate  a  priori  numerical  bounds  are  not  always  available.  However,  if 
the  existence  of  a  fixed  point  is  known  in  advance  from  say,  physical  con¬ 
siderations,  the  boundedness  requirement  on  C  can  be  dropped.  Our  im¬ 
mediate  objective  therefore  is  to  reach  Theorem  3.4  (Opial)  and  to  accom¬ 
plish  this  we  need  three  important  preparatory  lemmas,  the  last  two  of 
which  we  also  owe  to  Opial  [  4  3  • 

Lemma  3.  1:  The  set  of  fixed  points  IT  of  a  nonexpansive  mapping  T 
with  closed  convex  domain  C  and  range  W  is  a  closed  convex  set. 

Proof.  Letx^=Tx^,  i  =  and  suppose  that  x^-*x.  Since  [x^  }cC 

which  is  closed,  xeC  and  Tx  is  well-defined.  Thus,  invoking  nonexpansivity, 

|| Tx-x ||  =  ||Tx-Tx.  +  x.-x||  <  2||x-x.||  -»  0  (3.25) 

so  that  Tx=x  and  «T  is  closed.  To  establish  convexity  we  need  a  very  useful 
inequality. 

For  any  pair  (x,y)  eC  we  have  the  easily  verified  identity 
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(3.26) 


li  x-y  j|  2-  8  TX- Ty  ||  2  =  4  Re(Px-Py ,  ( 1  -P)x-(  1  -P)y) 


where  1  is  the  identity  operator  and 


(3.27) 


But  T  is  nonexpansive  hence, 

Re(Px-Py,(l-P)x-(l-P)y)  >  0  (3.28) 

for  every  x, yeC.  Since  P  and  T  have  precisely  the  same  fixed  points  it 
suffices  to  show  that  the  set  of  fixed  points  of  P  is  convex. 

Let  y  be  any  fixed  point  of  P.  Then  (3.28)  reduces  to 

Re(Px-y.x-Px)  >  0  (3.29) 


for  all  x€C.  Conversely,  if  some  yeC  satisfies  (3.29)  for  every  xeC  it 
satisfies  it  for  x=y  which  implies  that  || y-Py  ||  <  0  or  y  =  Py.  In  short,  the 
set  of  fixed  points  of  T  is  the  set  of  all  y  eC  that  satisfy  (3.  29)  for  all  xeC. 
But  this  set  is  obviously  convex,  Q.  E.  D. 

Corollary:  The  map  T  with  closed  convex  domain  C  is  nonexpansive 
iffP=(l+T)/2  satisfies  (3.28)  for  all  x,yeC. 

Definition  3.4:  A  map  T:  C-*  M  is  said  to  be  demiclosed  if  from 

{xn)cC  .  x„*>x0  »  xoeC  »  Txn"*yo  <3*30> 

4 

follows  Txg  =  yg. 

Definition  3.5:  A  map  T:  C-*C  is  said  to  be  asymptotically  regular 
if  for  every  x£C,  Tnx-Tn*  *x -*()>. 

Lemma  3.2:  In  a  Hilbert  space  h  let  the  sequence  [x^  }  converge 
weakly  to  Xq.  Then,  for  any  x/xq, 

^In  words,  T  is  demiclosed  if  for  any  sequence  {xn  )cC  which  con¬ 
verges  weakly  to  xqCC,  the  strong  convergence  of  the  sequence  (Txjto 
y0  implies  that  TxQ  =  yQ. 
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lim  inf  ||  x  -x  [|  >  lim  inf  ||  x  -x  | 
n-»eo  n-»<»  w 


(3.31) 


Proof.  Since  a  weakly  convergent  sequence  is  bounded,  both  limits 
in  (3.31)  are  finite.  Thus,  to  prove  this  inequality  it  suffices  to  note  that 
in  the  estimate 


llxn'xB  =!lxn"x0+x0"xll  2=  BXn”xoH  2+llx0-XH  Z+2Re(xn-x0>  xQ-x) 


(3.32) 


the  last  term  tends  to  zero  as  n-»co,  Q.E.  D. 


Lemma  3.3:  Let  T  be  any  nonexpansive  map  with  closed  convex 
domain  CcM.  Then,  1-T  is  demiclosed. 

Proof.  Let  {x^J^C  converge  weakly  to  x^  and  let  (x^-Tx^}  con¬ 
verge  strongly  to  y^.  Then,  since  T  is  nonexpansive, 

liminf||x  -x.  ||  **•  lim  inf  ||  Tx  -Txrt||  =  lim  inf  ||  Tx  -x  +x  -Tx  ||  = 

.  ^  11  n  0 "  —  ^  n  0 11  ^  11  n  n  n  0  " 


n-*  oo 


n  ->  oo 


=  lim  inf  ||  x  -y  -Tx  ||  >  lim  inf  ||  x  -x  | 
n-»»  n-»oo 


(3.33) 


by  lemma  3.2.  Hence,  again  by  lemma  3.2,  x^  =  y^+Tx0  or  (l-T)Xp  =  yQ 
so  that  1-T  is  demiclosed,  Q.E.  D. 

Theorem  3.  4:  Let  T:  C-*C  be  an  asymptotically  regular  nonexpansive 
map  with  closed  convex  domain  CCW  whose  set  of  fixed  points  «TcC  is  non¬ 
empty.  Then,  for  any  xeC,  the  sequence  (Tnx  J  is  weakly  convergent  to 
an  element  of  Z. 

Proof.  For  every  yeU,  the  sequence  {dR  j  =  {||Tnx-y||  }  is  non¬ 
increasing  because 

dn+ 1  =  II Tn+  lx-y  II =  II T(Tnx)  -  Ty  II  <  II  rtlx-y  B  =  dn  •  <3  •  34> 


Thus,  the  nonnegative  limit 


d(y)  =  lim  ||  T  x-y  | 
n-»  oo 


exists  as  a  finite  number  for  every  yCff. 


(3.35) 
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According  to  lemma  3.  1,  IT  is  a  closed  convex  subset  of  C  and  it 
follows  that  for  any  fixed  6>0,  the  set 

7b  =  (yeff:d(y)<6)  (3.36) 

is  a  closed  bounded  convex  subset  of  S’  which  is  nonempty  for  6  large  enough. 
Indeed,  convexity  and  closure  are  obvious  from  (3.35),  and  boundedness  is 
implied  by  the  inequalities 

II V 1  =  l|y-Tnx+  Tnx||  <  [| Tnx-y ||  +  ||Tnx«  (3.37) 


and 


|fTnx|j=  |lTnx-y0  +  y0|j<I|Tnx-y0||+  ||y0||  ,  (3.38) 


where  y^  is  any  preselected  member  of  S’.  Explicitly,,  for  any  yeffg 


I  y  II  <6+d(y0)+  II  y( 


(3.39) 


Since  bounded  closed  convex  sets  are  weakly  compact  (Theorem  2.  1, 

Corollary  3),  the  intersection  of  all  such  nonempty  ff^'s  is  a  nonempty 

closed  bounded  convex  set  Sc  .  Clearly,  6n  is  the  smallest  value  of  6  for 

°0  u 

which  Sg  is  nonempty.  The  set  Sg  can  contain  only  one  element  say  y^, 
for  if  we  suppose  that  it  also  contains  y^  ^y^,  the  identity 


I  Tnx 


yo+y  1  2 
— 7T 


=  II  Tn*-yn  II 2  +  II  Tnx-y .  || 2)  - 


(3.40) 


y0+yl 

yields  d( - ^ — )<6g  contradicts  the  meaning  of  6q. 


The  sequence  {Tnx}  converges  weakly  to  y^.  In  fact,  since  this 
sequence  is  bounded,  it  suffices  to  prove  that  all  possible  weak  limits  of 
its  various  subsequences  equal  y^.  ^  Let  Tnx-»y^yQ.  Then,  from  the 


- 5 -  n 

Assume  that  such  is  the  case  but  T  x  y£yg.  Then,  for  some  feM  it  is 

true  that  the  sequence  {(Tnx,f)3  fails  to  converge  to  (yg»f).  Hence,  there 
exists  a  subsequence  (Tnx)  of  (Tnx)  such  that  lim(Tnx,f)  exists  and  is 

j./ 

unequal  to  (yQ,f).  But  by  hypothesis,  the  sequence  {T  x}  itself  contains 
a  subsequence  {T  x}  converging  weakly  to  y0;  i.e.,  (yQ,  f)  =  lim(Tnx,  f)  = 
lim(Tnx,  f)  ^(yQ,f),  a  contradiction. 
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asymptotic  regularity,  of  T, 

Tnx  -Tn  +  1x=(l-T)Tnx  -*<j>  (3.41) 

and  invoking  the  demiclosed  character  of  1 -T  (lemma  3. 3),  ( 1  -T)y ^  =  d» ;  i.e., 
y^  is  a  fixed  point  of  T.  By  lemma  3.  2, 

60=1im||Tnx-y0||  >  lim||Tnx-yi  ||  =  dfy^  (3.42) 

which  is  incompatible  with  the  meaning  of  6q.  Thus,  for  every  xeC,  the 
sequence  (Tnx)  is  weakly  convergent  to  a  fixed  point  of  T,  Q.E.  D. 

Corollary:  The  sequence  (Tnx)  converges  strongly  to  y^  iff  at  least 
one  of  its  subsequences  converges  strongly. 

Proof.  We  know  that  Tnx  -»  y^,  a  fixed  point  of  T,  for  every  choice 
of  x€C.  Clearly,  since  the  weak  and  strong  limits  of  a  sequence  must 
coincide,  the  only  possible  strong  limit  of  any  subsequence  of  (Tnx)  is 
y^.  Now,  from  the  equality 

l|Tnx-y0||2=  |Tnx||2  -2Re(Tnx,y0)+  ||y0||2  (3.43) 

we  find  that 

Urn  |l  Tnx  ||  2  =  d2(y0)  +  ||  y Q  || 2  (3.44) 

/ 

asn-»«>.  In  particular,  for  any  subsequence  {t"x)  of  (Tnx), 

lim  ||  Tn  x  ||  2  =  d2(y0)  +  ||  yQ  ||  2  .  (3 . 45) 

/ 

But  as  we  have  already  remarked,  if  any  subsequence  {T  x}  converges 
strongly,  it  converges  strongly  to  y^  and  therefore  lim||T  x||  =  H y q H  ^  so 
that  necessarily,  d2(yg)  =  0  and 

Tnx  yQ  .  (3.46) 

Q.E.D. 

This  "all  or  nothing"  aspect  of  strong  convergence  exhibited  by  the 
iterated  sequence  (Tnx)  appears  to  be  the  rule  at  the  present  level  of 
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generality.  Unfortunately,  the  lack  of  strong  convergence  is  a  real  obstacle 
from  a  numerical  standpoint,  and  a  more  in-depth  analysis  is  undertaken 
in  the  next  section. 


Definition  3.6:  A  mapping  T:  C-*C  is  said  to  be  a  reasonable  wanderer 
if  for  every  xeC 

£  ||Tnx-Tn+1x||2<oo  .  (3.47) 

n=0 


It  should  be  evident  that  a  reasonable  wanderer  is  automatically  asymptotically 
regular  (Definition  3.5). 

An  example  of  a  nonexpansive  operator  that  is  not  asymptotically 
regular  is  easily  constructed.  On  the  real  line  choose  C=  [- 1 ,  1  ]  and 
define  T  by  Tx=  -x.  Clearly,  T  is  nonexpansive,  maps  C  into  C  and  its 
only  fixed  point  is  x  =  0.  However,  Tnx  -  Tn**x=  2(-l)nx  ■/*  0  as  n  -»oo  un¬ 
less  x=  0.  Nevertheless,  Browder  and  Petryshyn  have  shown  that  the 
convex  linear  combination 

Ta=ai  +  (l-a)T  (3.48) 


of  T  and  the  identity  operator  1  is  a  reasonable  wanderer  for  all  a  in  the 
interval  0<  a<  1  and  any  nonexpansive  T. 


Theorem  3. 5 [5  ]  :  Let  T:  C*+C  be  a  nonexpansive  map  with  closed 
convex  domain  C  whose  set  of  fixed  points  is  nonempty.  Then,  for  any 
fixed  a,  0<a  <  1,  T^:  C-*C  is  a  reasonable  wanderer  and  the  sequence 
{Tnx}  converges  weakly  to  a  fixed  point  of  T  for  every  xCC. 


Proof.  Clearly,  TR  maps  C  into  C  so  that  the  iterates  xn=Tflx 
are  well-defined  for  all  xCC.  In  addition,  for  1  the  fixed  points  of  T 
and  Tft  coincide  and  it  only  remains  to  prove  that  T^  is  a  reasonable 
wanderer  since  it  is  obviously  nonexpansive.  Let  yfiC  be  a  fixed  point 
of  T.  Then,  y  =  Ty  =  T^y  and 


ll*n+ry|l2=  Maxn+(1-a)Txn-yl|2=  I|a(xn-y)  +  (l-a)(Txn-y)||2  (3.49) 

=  a2||xn-y|2+2a(l-a)Re(xn-y,Txn-y)  +  (l-a)2||Txn-y!l2  . 

(3.50) 
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+•  .  f 


Similarly, 


8xn-TxnH  =  llxn-y-<Txn-y>ll2:=  llxn-y|l2“2Re(xn-y.Txn-y)+ llTxn-i'll 

(3.51) 

which  after  multiplication  by  a(l-(X)  hnd  addition  to  (3.  51)  yields 

llxn+j-y  II  2  +  ft(l -a)  II xn-Txn |]  2 -  a  || xn-y  ||  2  +  ( 1  -a)  || Txn-y  II  2  <  ||xn-y  || 2  . 


Hence, 


a ( 1  -a)  || xn-Txn  ||  2  <  ||  xn-y  ||  2  -  || xn+ 1  -y  II  2 


(3.52) 

(3.53) 


so  that 


N  N 

1 2  a  ii  _  „n+l  m2 


*0-a)ElvT*J  -AS  «T”x-T"+ixr£||x-yr-||xN+ryr<||x-y|| 

n=0  n=0 

(3.54) 

Consequently,  since  CL  -f  0, 

co 

i>n_.  »*.n+l„  ii  2 


Sl|T"x-Tn+ix|r<«,  . 


(3.55) 


n=0 


Q.E.D. 


For  ft  =  l/2,  Tjj  =  (1+ T)/2  in  which  case  Theorem  3.  5  is  easily 
deduced  from  the  inequality  (3.28). 
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4.  Iterative  Techniques  for  Image  Restoration  in  a  Hilbert  Space  Settin; 


According  to  the  discussion  initiated  in  the  introduction,  there  exists 
a  class  of  image  restoration  problems  in  which  the  unknown  f  can  be  as¬ 
certained  to  lie  in  an  intersection 

m 

c_  =  n  c.  (4.i) 

u  i=l 

of  well-defined  closed  convex  sets  C^,  i  =  l-»m.  Evidently,  Cq  is  also  a 
closed  nonempty  convex  set  containing  f  and  we  shall  denote  the  respective 
projection  operators  projecting  onto  Cq  and  C.,  by  and  P^,  i=  l-»m. 
Clearly,  f  is  a  fixed  point  of  PQ  and  all  the  PJs.  More  generally,  f  is  a 
fixed  point  of  Pq  and  of  every 

T.  =  1  + X.(P.-l)  (4.2) 

ill 


for  any  choice  of  relaxation  constants  Xj,  X^,  . ..,  X  .  Hence,  under 
the  same  conditions,  f  is  a  fixed  point  of  Pq  and  the  composition  operator 


T 


=  T  T  , 
m  m-1 


(4.3) 


Conversely,  can  we  say  that  any  fixed  point  f  of  T  lies  in  Cq  and  does  the 
sequence  {Tnx]  converge  (weakly  or  strongly)  to  a  point  of  Cq  as  n  tends 
to  infinity,  irrespective  of  the  initialization  xeW? 

Theorem  4.  1 :  Let  CQ  be  nonempty.  Then,  for  every  xeW  and  every 

choice  of  constants  X.,X_,  ...,X  in  the  interval  0  <  X  <  2,  the  sequence 

i  2  m 

(Tx  }  converges  weakly  to  a  point  of  Cq. 

Proof.  For  X.>0,  every 

T.  =  l  +  X.(P.-l)=(l-X.)l+X.Pi  (4.4) 

is  nonexpansive .  The  assertion  is  obviously  correct  for  0<X^<1,  but  if 
1  <  X^  we  have  1-X.  <0  and  it  is  necessary  to  reason  differently.  With  the 
aid  of  (2.27)  and  (2.30),  it  is  found  that 


|| T.x  -  T.y  ||  2  =  || { 1  - X.)(x-y)  +  X.(P.x  -  P. y)  ||  2 


=  (l-Xi)2||x-y||2  +  2Xi(l  - X.)Re(x-y,  P.x-P.y)  +  X2  j|  P.x-P.y  ||  2 

<  (l-Xi)2|x-yI2  +  (xf+2Xi(l.Xi))|Pix-Piy|2 

=  (l-Xi)2||x-y||2+X.(2-X.)||Pix-P.y||2 

<  (Xi(2-Xi)  +  (l-Xi)2)||x-y||2=  ||x-y||2  (4.6) 

and  nonexpansivity  is  established.  We  now  show  that  T  is  a  reasonable 
wanderer. 

For  m=l,  T=Tj,  Cq=C^  and 

||x-Tx||2=X2||x-P1x||2  .  (4.7) 

Moreover,  for  any  yeC^,  Ty=P^y=y  and 

||Tx-y||2=  ||x-y+X  j(P^x-x)  || 2  (4*8) 

=  ||x-y  ||  2  +  2  X  jRe(x-y,  P  jX-x)  +  XjUx-PjxH2 

=  I|x-y||2-X1(2-X1)  j|x-P1x||2  +  2X1Re(x-P][x,y-P1x) 

(4.  8a) 

<  ||x-y  ||  2  -  X  j(2-  X  j)  ||  x-PjX  ||  2  (4.9) 

because  the  last  term  in  (4.8a)  is  nonpositive.  Thus,  by  combining  (4.7)  and 
(4.  9)  it  is  found  that 

||x-Tx||2<  ^L(||x-y||2-  ||Tx-y||2)  (4.  10) 

for  0  <  X  j<  2. 

For  arbitrary  m  >  1,  a  straightforward  induction  on  m  yields  the 
inequality 
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II*-Tx||2<  bm.  2m-1(||x-y||2  -  ||Tx-y||2)  (4.11) 


where 


X. 


bm=  "j 

1<  Km  1 


(4.12) 


(Clearly,  (4.  10)  subsumes  the  case  m=  1.)  In  fact,  let  T  =  T  K  where 

m 


K  =  T  ,T  ,  ...  T. 
m  - 1  m-2  1 


(4.13) 


and  observe  that  for  m  >  2, 

||x-Tx]]2  =  ||  x-Kx+Kx,-  Tx  J|  2 

<  (||x-Kx||+  ||Kx-Tx||)2 

<  2(  ||  x-Kx  ||  2  +  ||Kx-Tx  ||  2) 

<  2(||x-Kx||2+ 2m‘2||Kx-TmKx||2)  .  (4.15) 

Thus,  by  the  induction  hypothesis,  ^ 

||x-Tx||2<bm.2(2m-2l|x-y||2.2m"2||Kx.y|l2+2m"2||Kx.y||2-2m-2||1r.x-y||2) 
=  bm.  2m ” 1  ( || x -y || 2- 1| Tx-y || 2)  ,  (4.16) 

the  desired  inequality. 

It  now  follows  immediately  from  (4.  16)  that 
oo 

£  II  Tnx-Tn+  ||  2  <  bm  Z™  - 1 1|  x-y  ||  2  <  oo  (4.17) 

n=0 

and  T  is  therefore  a  reasonable  wanderer  and,  a  fortiori,  asymptotically 
regular.  By  Theorem  3.4,  the  sequence  (Tnx)  converges  weakly  to  a  fixed 
point  of  T. 


m 


and  b  > 
m  — 
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However,  the  fixed  points  of  T  coincide  precisely  with  the  points  of 
Cq,  the  intersection  of  the  C^'s.  Indeed,  it  is  obvious  that  xeC^  implies 
x=Tx  since  xCC.,  i=l->m.  Conversely,  if  x  =  Tx  and  y 6Cq, 

llx-y||  =  llTx-Ty|l  <  llT1x-T1y||  =  ll^x-yll  £  llx-y|j  (4.1 8) 

hence,  ||x-y||  =  ||TjX-y||.  In  view  of  (4.  10)  this  is  only  possible  if  x  =  TjX 
so  that  x=  j  •  l^x  ant*  a  rePet*ti°n  °f  the  argument  finally  leads  to 

xC(L ,  i=l-»m;i.e.,  xgCq  and  the  proof  is  complete,  Q.  E.  D.  ® 

We  are  now  in  a  position  to  address  the  question  of  strong  convergence. 
Let  T:  C  -»C  denote  a  nonexpansive  asymptotically  regular  operator  with  closed 
convex  domain  Cch  whose  set  o  of  fixed  points  is  nonempty.  From  the 
corollary  to  Theorem  3.4,  the  sequence’  (Tnx3  converges  strongly  to  a 
member  of  IT  iff  at  least  one  of  its  subsequences  converges  strongly. 

Clearly,  such  is  assured  if  all  iterates  Tnx  lie  ultimately  in  some  compact 
or  finite-dimensional  subset  of  M. 

The  latter  possibility  exists  even  if  C,  the  domain  of  T,  is  itself  not 
compact  as  is  seen  in  the  case  of  the  composition  operator  T  of  Eq.  (4.3) 
with  all  X^s  equal  to  one.  Here,  C=H,  'I\  =  P. ,  i=  l-*m, 

T  =  P  P  ,  •  P.  (4.  19) 

m  m  - 1  1 

and  alliterates  Tnx,  n>l,  lie  in  C  .  It  suffices  therefore  that  C  be  com- 

—  q  m  m 

pact  or  finite -dimensional.  (Assumptions  of  this  kind  appear  to  be  quite 
natural  for  those  digital  restoration  problems  falling  within  the  scope  of 
Theorem  4.  1. ) 

It  is  interesting  and  important  to  note  that  if  all  C^'s  in  Theorem  4.  1 
are  CLM's,  the  sequence  of  iterates  {T  x}  not  only  converges  strongly 
but  actually  converges  to  the  projection  of  x  onto  Cq.  This  result  is  an 
extension  of  I.  Halperin's  neat  generalization  of  Von  Neumann's  celebrated 
alternating  projection  theorem  [6]  from  m  =  2  to  arbitrary  [7  1. 


8 

That  a  composition  of  appropriately  relaxed  projection  operators  is 
automatically  a  reasonable  wanderer  is  of  course  directly  attributable  to 
the  special  properties  of  projections  onto  closed  convex  sets. 

Q 

7By  this  we  mean  that  C__  is  contained  in  a  finite -dimensional  linear 
manifold  c  M, 


25 


Corollary  1:  Under  the  constraints  of  Theorem  4.  1,  if  every  C.  is 
a  CLM,  T°  converges  strongly  to  P^,  the  orthogonal  projection  operator  on¬ 
to  the  CLM  Cq. 

Proof.  ^  Let  us  first  remark  that  now  all  P.'s  and  P„  are  bounded 

i  0 

linear  self-adjoint  operators.  Consequently, 

T*  =  1+  X  (P*-l)  =  1+  X.(P.-l)  =  T  (4.20) 

i  i  i  '  lit 

is  self-adjoint  and  linear,  i  =  1  -»m,  and  T°  is  also  linear  for  all  n  >  0.  More¬ 
over,  since  T  i.s  nonexpansive  and  T<j>  =  (}>, 

It  Tx  {I  =  ||  Tx-T<J>  [|  <  ||x-<j>  ||  =  |x|  (4.21) 

for  all  xeU  so  that  || T  |j  and  therefore  || Tn ||  <1,  n>  0. 

As  has  already  been  shown  in  Theorem  4.  1,  for  every  xtU, 

Tnx  -  Tn+ix=  Tn(l-T)x-»<J>  (4.22) 

hence,  Tny-*4>  f°r  aH  Y  111  R(l-T),  the  range  of  1-T.  Now  [l0]the  closure 
[P.(l-T)  j  of  the  range  of  the  bounded  operator  1-T  is  the  orthogonal  comple¬ 
ment  of  the  null  space  of 

“•T)*s,-<TmTm-l-T/S  1-T1T2‘*-Tm  •  <*•  23> 

from  which  it  is  deduced  that  x  belong  to  this  null  space  iff 

x  =  T  T_  .  .  .  T  x  .  (4.24) 

12m 

Thus,  in  view  of  Theorem  4.  1,  this  null  space  is  identical  with  Cq  so  that 
by  the  projection  theorem,  h  =  Cq  +  [R(  1  -T)  ]  . 

Let  ye[R(l-T)].  Then,  there  exists  a  sequence  (y.  JcR(l-T)  such 
that  y^->y.  Clearly, 

!|Tny||  ^  ||Tnyi  +  Tny -Tny.||  <  ||Tny.  ||  +  Hy-yJI  ,  (4.25) 

n>l,  which  immediately  gives 

^Since  i=  l"»m,  4>eCQ  which  is  therefore  never  empty. 
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(4.26) 


lim  sup||Tny  ||  <  ||y-y.|| 

n-»oo 

because  Tny^ -»$  for  every  fixed  i:>l.  But  ||y-y^|| -»0  so  that  Tny-»<i>  as  n-»oo 
for  every  ye[R(l-T)].  Finally,  again  from  the  orthogonal  projection  theorem 
every  xeM  admits  a  unique  decomposition  x  =  z+y  where  z€Cq  and  ye[R(l-T)3. 
Consequently,  Tnz  =  z  for  all  n>  1  and 

Tnx=Tnz  +  Tny  =  z  +  Tny -+z  =  PQx  (4.27) 

as  n-»°o.  Therefore,  by  definition,  Tn  converges  strongly  to  the  orthogonal 
projection  operator  onto  Cq,  Q.E.D. 

Corollary  2:  If  each  is  a  linear  variety,  *  *  i  =  l-*m,  and  if  the  in¬ 
tersection  Cq  of  these  varieties  is  nonempty,  the  sequence  (T  x}  converges 
strongly  to  P^x  for  every  xeW. 

Proof.  To  avoid  unnecessary  notational  complications  we  assume 
that  m  =  3.  Thus,  C.  =  e.  +  8.  where  g.  is  a  fixed  element  of  h  and  8.  is  a 

’  l  6i  i  bi  l 

given  CLM,  i=  l-»3.  It  is  easily  shown  that  for  each  i  and  any  xeM, 

P.x  =  S.x  +  R.g.  (4.28) 

l  r  i°i 

where  is  the  orthogonal  projection  operator  onto  8^  and  R^is  that  onto  i*s 
orthogonal  complement,  J.8^  Clearly  then, 

Tix=x  +  X.(S.x+R.gi-x)  =  (l-X.)x+  X.S.x  +  X.R.g.  (4.29) 

=  L.x  +  X.R.g.  (4.30) 

l  l  i°i 

where 

L.  =  (l-X.)l  +  X.  S.  ,  i=l-»3  .  (4.31) 

1  '  l'  1  1 


Thus,  by  simple  algebra  and  linearity, 


Tx=TjT2T1x=  X3R3g3+  X2L2R2g2+  X1L3L2R1g1  +  L3L2L1x 


=  h  +  Lx 


(4.32) 


where 

A  linear  variety  C  is  composed  of  all  vectors  x  of  the  form  x  =  g+y 
where  g€h  is  fixed  and  y  ranges  throughout  some  CLM  8.  For  short, 

C  =  g+8,  (A  linear  variety  is  obviously  closed  and  convex.) 


(4.33) 


and 

L  =  LjI^Lj  • 

It  now  follows  by  iteration  of  (4.  32)  that 

n-1 

Tnx  =  J  Lrh  +  Lnx  ,  n>  1 
r=0 

In  particular,  for  x=f,  fa  fixed  point  of  T,  Tnf  =  f, 

n-1 

f  =  Yj  Lrh  +  Lnf 
r=0 


(4.34) 


(4.35) 


(4.  36) 


and 


Tnx  -  f  =  Ln(x-f)  ,  n  >  1 


(4.37) 


According  to  corollary  1,  Ln(x-f)  -»Pg(x-f)  hence, 


T  x  -*f+  Pg(x-f)  s  w 


(4.38) 


where  Pg  is  the  orthogonal  projection  operator  projecting  onto  the  CLM 


s  =  n  s.  . 

i=l  1 


(4. 39) 


Obviously,  w  =  limTnx  is  independent  of  the  choice  of  feCg. 


n-»oo 


It  is  clear  from  feCg  that  weCg,  and  invoking  the  orthogonal  pro¬ 
jection  theorem, 

x-w  =  x-f-Pg(x-f)  =  Q  g(x-f)  (4.39a) 

where  Qg(x-f)  is  the  projection  of  x-f  onto  aS,  Hence 

|| x-w  ||  =  ||Qg(x-f)||  <||  x-f ||  .  (4.39b) 

But  f  is  an  arbitrary  member  of  Cq  so  that  necessarily,  Tnx  -*w=  Pqx»  Q.E. D. 


- 1 7  “  - - - — 

Another  proof  of  corollary  2  (without  relaxation)  is  given  by  I.  Atnemiya 
and  T.  Ando  [8  3. 
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In  Theorem  4. 1  the  update  rule  is  given  by  x  ,  =  Tx  with  x.  =  x  an 

°  7  n+ 1  n  0 

arbitrary  element  of  U.  More  generally,  we  can  also  work  with  the  single- 
step  cyclic  scheme 


=  V 

=  T2X1 


x 


m 

Xm+1 


=  T  x  ,  =  T  T  .  . .  .T.x  =  Tx 
m  m-1  m  m-1  1 


=  T.x 
1  m 


(4.40) 


x2m 

x2m+l 


=  T 
=  T 


mX2m-l 

lX2m 


=  T  T  . 
m  m-1 


•  •  •  T  jTx  = 


T 


2 

x 


Or,  put  more  succinctly, 

Xn+1  ”  ^a(n)Xn 


(4.41) 


where  Xq  =  x  and^ 

tt(n)  =  1  +  nmodm  ,  n>0  ..  (4.42) 

It  is  easily  shown  that  the  sequences  {Tnx}  and  {x^}  converge  weakly  to 
the  same  limit  x*€Cq.  Write  n=fm  +  r,  0<r<m-l.  Then,  for  n>  1 


’cn  =  TrTr-l-’-TlT  * 


(4.43) 


and  i  -» so  as  n  ■* eo.  Let  T^x  -*x*  and  let  the  subsequence  (x  /  }  of  {x  } 

n  n 

converge  weakly  to  x  , 

Then,  n#=mf/+r/,  0<  r'<  m-1,  and  «  as  n^w.  Moreover, 

—  —  i • 

making  use  of  (4.  1 1)  with  T  replaced  by  T^ /Ty /_ ^  . . .  Tj  and  x  by  T  x  we 

- n - 

n  mod  m  is  the  remainder  obtained  upon  division  of  n  by  m  and  is  one 
of  the  integers  0,  1,  . .  .,m-l. 
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(4.  44) 


obtain,  for  every  y€Cg, 

llT<  x-xn,  ||2<bm-  2m'1(||T/ x-y||2-||xn/-y||2) 

<bm .  2m  " 1  ( 1|  T1  x-y  ||  2  -  1 T1  /+  ^-y  1  2)  .  (4.  45) 

l'  I*  14 

Hence,  ||T  x-x  / 1|  -» 0  as  n/ -» oo.  But  TZ  x-x  /-»  x#-x*and  since1 
"  n  "  n 

||x*-x'||  <  liminfJlT*  x-xn<  ||  =  0  ,  (4.46) 


n  -» oo 


x#  =  x/,  as  was  to  be  shown.  (To  the  best  of  our  knowledge,  the  first  proof 

of  weak  convergence  of  the  iteration  (4.41)  to  a  point  of  Cq  is  due  to 

Bregman  [9].  However,  as  we  have  seen,  the  underlying  theory  is  the 

same  as  that  of  the  full  composition  operator  T=  1  ’ "  *^1  ant* 

associated  fixed-point  recursion,  x  .  ,  =  Tx  . ) 

n+ 1  n 

The  first  lemma  of  this  section  is  preparatory  to  a  deeper  study  of 
the  strong  convergence  of  the  sequence  (Tnx). 


Lemma  4.  1:  Let  T  be  any  nonexpansive  map  whose  set  of  fixed 
points  includes  a  given  closed  convex  set  »T,  and  let  denote  the  pro¬ 
jection  operator  onto  Z.  Then,  for  anyxS  domain  T, 

Re(x-Tx,  x-Pyx)  >  0  (4.47) 

and 

||x-Txfl  <  2||x-P,j,x||  .  (4.48) 


*^Let  yn  -»  f.  Then, 
|(f.y)|  =lim|(yn,y)|. 
so  that 


for  every  yeW,  (f,y)  =lim(yn,  y)  as  n-*«>.  In  particular. 
But  from  Schwartz's  inequality,  |(yn»y)|  <  ||y||«  l|yn|| 

|(f.y)|  <  l|y||  *  lim  inf  1| y  l|  . 

.  n-»°o 


Thus,  for  y  =  f , 

||f||<liminf||y  1  . 

n-»oo 
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(4. 49) 


Proof.  Clearly,  for  every  x,  y  in  domain  T, 

»T*-Ty||2-||*-y||2<0  . 

In  particular,  if  y  =  P^x, 

llTx-Pffxl|2<llx-pffxl|2  (4.50) 

because  P^xe?  and  TPjX=Pj,x,  by  hypothesis.  Thus, 

|jx-Tx-(x-Pffx)  j|2<  ||x-Pj,x||Z  (4.51) 

and  an  easy  simplification  yields 

JJx-Txj|  Z  <2  Re(x-Tx,x-Pyx)  ,  (4.52) 

x€  domain  T.  From  (4.52)  and  Schwartz's  inequality  we  now  obtain  (4.47)  and 
(4.48)  immediately,  Q.E.O. 

Inequality  (4.47)  simply  states  that  Tx  lies  on  the  side  of  the  support 
plane  through  x,  parallel  to  that  through  P^x,  containing  7  (Fig.  4. 1). 


Lemma  4.2:  The  iterated  sequence  {lnx}  of  Theorem  3.4  converges 
strongly  iff 

|lTnx-PjyTnx||-*0  (4.53) 


as  n-»«. 
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^Clearly,  |Tnx-PffTnx|  =  d(Tnx,  7),  the  distance  of  Tnx  from  7. 


Proof.  As  shown  in  Theorem  3.4,  the  sequence  {Tnx}  converges 
weakly  to  some  point  x*ej.  Evidently,  if  the  convergence  is  actually  strong, 


d(Tnx,  S)  =  ||Tnx-PjyTnx||  <  ||Tnx-x*||  -*  0  (4.54) 

K 

as  n-*oc.  Conversely,  suppose  that  d(T  x,  *T)  -*0.  Then,  since  T  is  non- 
expansive  for  all  integers  k>0  and  includes  IT  in  its  sets  of  fixed  points, 

(4.  48)  yields,  for  all  x€C, 

[I  x-Tkx  |1  <  ZlIx-PjxH  ,  k>  0  .  (4.55) 

In  particular,  if  x  is  replaced  by  Tnx  we  obtain 

||Tnx-Tn+kx||  <  2||Tnx-P;rTnx||  -»0  (4.56) 

as  n-»°°.  Thus,  the  sequence  {Tnx}  is  Cauchy  and  T°x  ->x*,  Q.E.  D. 

According  to  lemma  4.2,  convergence  in  Theorem  4.  1  of  the  sequence 
{Tnx  }  to  a  point  of  Cq  is  strong  iff 

d(Tnx,C0)  =  ||Tnx-P0Tnx||  -*0  (4.57) 


as  n  oo.  (The  set  of  fixed  points  of  T  =  T  T  . .  .  T .  coincides  with  Cn.  ) 

m  m  - 1  i  u 

We  can  show  without  the  imposition  of  any  additional  constraints  that  for 
r  =  1  ->m, 


d(Tnx,Cr)  =  ||Tnx-PrTnx||  -0 


(4.58) 


as  n-**.  (The  distance  of  T  x  from  every  goes  to  zero  as  n  goes-to 
infinity. ) 


To  see  this,  note  that  as  a  special  case  of  (4.  11), 
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r  r- 1  1 


|Tnx-P_P_  .  ...P,Tnx||Z<  2r-1(||Tnx-y||2-||PrPr_1  ...PjT^-yH2) 

(4.59) 

<  2r'1(||Tnx-y||2.  ||Tn+1x-y||2)  -0  (4.60) 


for  all  yCC^  and  every  r  =  1  -»m  because , 

^Identify  T  with  PrPr  ^  .  .  .  P^  and  set  m  =  r  and  =  1 ,  i  =  1  -*  r. 
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i|Tn+1x-y||  <  ||PrPr_1---PlTnx-y||  (4.61) 

and 

lim  j|Tnx-y||  (4.62) 

n-»oo 

exists  (Theorem  3.4).  Thus,  since  P^P^  j...PjTnx£Cr  , 

d(Tnx,  Cr)  <UTnx-PrPrl  ...PjT^I  -*0  (4.63) 

as  n-»oo  and  (4.58)  is  established.  Our  problem  therefore  is  to  find  some 
other  restriction,  preferably  very  weak,  which  together  with  (4.  58)  implies 
(4.57).  The  next  theorem  complements  Theorem  4.  1  and  supplies  two  suf¬ 
ficient  conditions  for  strong  convergence,  the  second  of  which  appeared  for 
the  first  time  in  the  excellent  paper  by  Gubin,  Polyak  and  Raik  [l  3  already 
referred  to  in  Theorem  2.3. 

Theorem  4.  2:*7  Either  of  the  two  conditions  1)  or  2)  stated  below 
suffices  to  guarantee  the  strong  convergence  of  the  sequence  {Tnx  }  of 
Theorem  4.  1  to  its  weak  limit  x?*. 

1)  At  least  one  of  the  C.'s  is  uniformly  convex  and  does  not  contain 
x*  in  its  interior. 

2)  For  some  a,  l<a<m,  the  set  intersection 

m  o 

can(nc.)  (4.64) 

i/a 

is  nonempty  (and  therefore  contains  a  point  of  Cq). 

In  case  2)  the  strong  convergence  of  {Tnx}  to  x*  is  at  a  geometric 

rate. 

Proof.  1)  To  be  specific,  let  us  suppose  that  is  uniformly  convex 
and  contains  xP  on  its  norm  boundary.  As  we  have  already  shown  in  (4.  58), 

d(Tnx,CL)  =  ||Tnx-P1Tnx ||  -»0  (4.65) 

'The  interior  of  any  set  C  is  denoted  byC°. 
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18  n 

asn-*<».  Clearly,  every  y  =P.T  x  lies  in  C,  and  y  -*-x*.  Now,  if  the 

n  i  in 

convergence  of  the  sequence  {y^  J  to  x*  is  not  strong,  there  exists  an  6>0 
and  a  subsequence  {yn /  3  such  that 

l|yn'  -**ll  >  e  (4.66) 

for  all  n'. 

Thus,  since  x^eC^cCjand  the  latter  is  uniformly  convex  (Def.  2.3), 
the  point 

y  /  +  x* 

V  =  ~JL~2 —  +  h  (4>  67) 

is  a  member  of  Cj  for  all  n7  and  all  h  such  that 

||h||  <  6(6)  >  0  .  (4.68) 

But  for  any  such  fixed  choice  of  h,  z^/  ->  x^+h  which  belongs  to  C ^  because 
the  latter  is  weakly  closed.  Consequently,  x#  is  the  center  of  a  sphere  of 
radius  6(e)  >0  contained  in  C^,  and  this  contradicts  the  assumption  that  x* 
is  not  an  interior  point  of  C^.  Therefore  the  convergence  of  {y^}  and 
hence  of  {Tnx}  to  x*  must  be  strong.  (Note  that  ||Tnx-x*||  <  ||  Tnx-yr  ||  + 
||yn~x*  ||  -*  0  as  n-»  oo. ) 

2)  Let  z  belong  to  the  intersection  (4.64).  Then,  zgCq  and  for  some 
6>0  it  is  true  that  he(L,  i/  a,  for  all  h  such  that 

Oh-z  I|  <  6  .  (4.69) 

Lety~PaTnx.  Then,  for  any  e>0,  no  matter  how  small, 

w=  efe ’z+e?6  *y  (4*70) 

is  a  member  of  Cq  for  all  sufficiently  large  n. 

In  fact,  since  zeC^  andyeC^and  C^is  convex,  weC^.  Now  observe 

that 

18 

For  any  zeW, 

|  (Tnx,  z)  -(yn,  z)  |  =  |  (Tnx-yn,  z)  |  <  ||  Tnx-yn  ||  •  ||  z  ||  -*  0 

as  n  ■»»,  Hence,  lim  (y  ,  z)  =  lim  (Tnx,  z)  =  (x*,  z)  so  that  y  -*  x*. 
n-»oo  n  n-»oo 
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(4.71) 


Hy-P.yll  =d(y,C.)  <||y-P.Tnx||  <||Tnx-yjj  +  ||  Tnx-P.  Tnx  || 
=  d{Tnx,  Ca)  +  d(Tnx,  C.)  <  |  +|  =  e 

19 

for  n  large  enough.  Thus,  for  all  such  n, 

h  =  z  +  -|  (y-P.y) 

belongs  to  C^,  i/a,  because  ||h-z||  <  6.  From  the  identity 

w  =  ele  (z  +  !(y-Piy))  +  ITT  '  piy  =  ITT  * h  +  ITT  ’  Piy  • 


(4.72) 

(4.73) 

(4.74) 


and  the  convexity  of  C  ^  we  also  conclude  that  w£C. ,  i  /  a  ;  i.  e.  ,  wC  C^,  i  =  1  -»m, 
hence  wcCq. 

Consequently, 


d(Tnx,  CQ)  <  ||  Tnx-w  ||  <  ||Tnx-y||  +  ||y-w|| 

(4.75) 

=  d(Tnx,  Cft)  +  elg-Hy-z  || 

(4.76) 

e  eR  _ ,  2R  e  _  ce_ 

<  t  +  -5 —  t1  +-g->  *  1  '  T 

(4.77) 

where 

c  =  d+f)  (4.78) 

and  R  is  an  upper  bound  on  ||y-z||.^  Since  C  is  arbitrary,  d(Tnx,  Cq)  con¬ 
verges  to  zero  as  n-*°o  and  {Tnx}  s  therefore  strongly  convergent  to  x*. 

It  remains  to  establish  that  convergence  is  at  a  geometric  rate. 

Observe  first  that  c/2  in  (4.  77)  is  by  construction,  and  for  sufficiently 
large  n,  an  upper  bound  on  the  m  distances  d(Tnx,  C^),  i  =  1  -* m : 

d(Tnx,  C.)=  ]|Tnx-P  Tnx||  <  t  ,  i  =  1-m  .  (4.79) 

1  I  “  ^ 

19  n 

Remember  that  according  to  (4.  58),  for  i  -  1  -*m,  lim  d(T  x,  C.)  =0. 

n  -»°o  k 

^®||y-z||  =  ||PaTnx-z||  <  ||Tnx-zj|  for  all  zeC q.  But  the  latter  is  a  monotone 
nonincreasing  function  of  n  and  is  therefore  bounded  by  some  finite  number  R, 
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Hence,  for  all  large  enough  n 


d(Tnx,  CJ  <  c  .  sup  ||Tnx-P.Tnx| 

i=  1-+*ij  1 


(4.80) 


Choose  the  positive  numbers  and  so  that  e^e^/ c  <1  and 

0<  e  <  X.  <  2-e  ,  i  =  1  -»m .  Then,  from  (4.  10)  and  (4.7)  with  T  replaced  by 
1 —  1  ~**r_  z 

1 

0* 

2 


T.,  x  by  Tnx  and  y  by  PnTnx  we  obtain,  for  every  i= 


d^(Tnx,  CQ)  <  cZ.  d^(Tnx,  Ci)  <  y-7’7TXT(d  (T“*.  C0>  “  |T.Tnx-P  Tnx|  ) 

i' 

(4.81) 


1  g^(dZ(Tnx,  CQ)  -  ||Tn+1x-P0Tnx||2) 
2 

<  (d2(Tnx,  C0)  -  d2(Tn+  1x,  CQ))  . 

1  2 


(4.82) 


(4.83) 


Therefore, 


n+1  ei£2  l/Z  n 

d(Tn+ix,  e0)<  (1  -  d(Tnx,  CQ) 

c 


(4.84) 


and  by  iteration, 


€  e_  n/2 

d(Tnx,  CQ)  <(1 - -j— )  -d(x,C0) 

c 

e  e  n/2 
<R(1  -  -L^) 


(4.85) 

(4.86) 


The  proof  can  now  be  brought  tc  a  quick  conclusion.  From  (4.  56),  with 
IT=  Cq  and  Pjj,  =Pq,  (4.  86)  yields,  for  every  integer  k>  1 , 

||Tnx-Tn+kx||  <  211(1  -  -~)  ^  .  (4.87) 

c 

But  for  fixed  n,  T  x-»x*  as  k-»°o  so  that  a  passage  to  the  limit  in  (4.87) 
gives 
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T  x-x*  <  2R(  l  - 


€lC  n/2 


(4.88) 


i.e.,  T°x  converges  strongly  to  Us  limit  x*  at  least  as  fast  as  2Ran  where 


e  e  1/2 

a  M  1  -  <1 


(4.89) 


Q.E.D 
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The  idea  behind  the  proof  of  1),  Theorem  4.2,  is  simple  but  the 
criterion  itself  is  very  effective.  It  differs  fundamentally  from  2),  and 
taken  in  the  context  of  Theorem  4.  1,  appears  to  be  new.  Apart  from  the 
important  uniform  convexity  requirement,  1)  succeeds  because  the  weak 
limit  x*  is  on  the  boundary  of  at  least  one  C.,  whereas  the  success  of  2) 
depends  entirely  on  the  fact  that  at  least  one  point  of  Cq,  not  necessarily 
x*,  is  interior  to  the  set  intersection  of  some  m-1  C.'s.  In  general,  with¬ 
out  the  introduction  of  smoothing  constraints,  2)  is  of  limited  applicability 
because  for  the  class  of  image  -  restoration  problems  we  have  in  mind,  the 
unknown  f  is  usually  a  boundary  point  and  convex  sets  with  interiors  are 
hard  to  come  by. 

For  example,  suppose  that  V-L^,  the  Hilbert  space  of  all  functions 
f(t)  of  the  real  variable  t  square -integrable  over  -oo<t<  oo.  Let  F(uu)  de¬ 
note  its  L^-Fourier  transform  and  let  us  denote  the  correspondence  by 
f(t)“— ■  F(u:),  Then,  if  f.(t)  ~  F.(u>),  i  =  1 , 2,  22 


00 


00 


<fj ,  f2)  =  f  =  J  F1(uu)F2(U))dO) 


<4.  90) 


For  any  prescribed  b>0  let  denote  the  subset  of  W  composed  of  all  f's 
bandlimited  to  b;  i.  e. ,  f €  iff  F(ttJ)  =  0,  |  U)  j  >  b.  It  is  obvious  that  is 
a  CJLM  devoid  of  any  interior  points.  Indeed,  given  any  e>0  and  any  feC^ 
there  exists  a  g/C^  such  that 


— n - 

Except  for  some  minor  improvements,  especially  in  the  part  dealing 
with  geometric  convergence,  our  proof  of  2),  Theorem  4.2,  is  in  all  respects 
the  same  as  the  original  one  given  by  Gubin  et  al.  [  1  3. 

22  — 

For  a  scalar  x,  x  denotes  its  complex  conjugate,  and  as  usual  Rex  and 
Im  x  are  abbreviations  for  the  "real"  and  "imaginary"  parts  of  x,  respectively. 
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oo‘ 

|U-g  tl  2  =  /|f(t)-g(t)|2dt  <  €2  .  (4.91) 

-ao 

(Thus,  as  measured  in  terms  of  energy  context,  any  size  neighborhood  of  a 
bandlimited  sig  ial  contains  signals  that  are  not  bandlimited. ) 

As  another  illustration,  for  any  prescribed  a  >  0,  let  C  denote  the  set 
of  all  f's  whose  orthogonal  projection  P  f  onto  the  closed  interval  -a<t<  a 
is  a  given  function  g(t).  (Of  course,  g(t)  =  0,  Jt]  >a.)  It  is  easily  seen  that 
C  is  a  linear  variety  whose  interior  is  also  empty.  Thus  C  and  C,  are 
two  practical  examples  of  boundary  sets  that  occur  repeatedly  in  many  areas 
of  signal  processing. 

Let  Cj  denote  the  closed  unit  sphere  in  W  and  let  g  be  any  fixed  element 

of  norm  equal  to  one.  Clearly,  g  lies  on  the  boundary  of  C . ,  a  closed  uni- 

23  1 

formly  convex  set.  Let  denote  a  closed  half«space  whose  boundary 

is  tangent  to  C  j  at  g.  Obviously,  g  is  the  only  point  common  to  the  inter¬ 
section  C  j  HC^.  To  find  g  we  can  generate  the  iterates  Tnx  where  T  = 
and  x  is  arbitrary.  (Figure  4.2  should  clarify  the  mechanics.) 


it  is  easily  shown  that  the  closed  sphere  of  radius  R  is  strongly  convex 
(Def.  2.  4)  with  6(t)  =  p and  (i  =  1/8R. 
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Since  x*  =  g  lies  on  the  boundary  of  C^,  criterion  1)  guarantees  that 
Tnx-»g;  however,  2)  fails  because  is  empty.  Nevertheless,  geo¬ 

metric  convergence  is  precluded  by  the  tangency  of  the  two  boundaries  at 
the  point  of  contact  x*. 

In  Fig.  4.3,  2)  always  succeeds  and  establishes  that  Tnx-*x*  at  a 
geometric  rate.  On  the  other  hand,  1)  predicts  strong  convergence  only 
if  Cj  is  uniformly  convex.  Neither  1)  nor  2)  encompasses  corollary  2, 
Theorem  4.  1,  in  which  all  C^s  are  linear  varieties. 
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